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DETAILED ACTION 
Continued Examination Under 37 CFR 1. 1 14 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on August 
24, 2004 has been entered. 

2. Applicants' remarks regarding Claims 1-40 have been fully considered but they 
are moot in view of new grounds of rejection. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 15-26, 38, 39, 45 and 46 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Saber et al (Face Detection and Facial Feature Extraction Using Color, 
Shape and Symmetry-based Cost Functions; IEEE Proceedings on Pattern 
Recognition, ISBN: 1015-4651). 

Regarding Claim 15, Saber et al disclose a method of extracting a specified 
image subject, comprising the steps of: 

performing image subject extraction processing by a specified image subject 
extracting algorithm or algorithms for each extraction area (Figures 1-3; Pages 656-657, 
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Section 2, Sub-sections 2.1-2.4, skin/non-skin classification and shape classification 
algorithms); 

performing a vote in an N-dimensional space of an image characteristic quantity 
for each extraction area extracted by said specified image subject extracting algorithm 
or algorithms (Figures 1-3; Pages 656-657, Section 2, Sub-section 2.5, cost function 
(5)); and 

performing weighting of degree of certainty as a specified image subject based 
on an aggregation value of the vote within a section area for aggregation in said N- 
dimensional space (Figures 1-3; Pages 656-657, Section 2, Sub-section 2.5, cost 
function Formula (5), weighted combination of the cost function) Formula (10)), 

wherein a first extraction area is extracted according to a shape of regular 
geometric form of the specified image object (Figures 2 and 3; Page 655, Section 2. 
Skin region has been extracted by fitting an ellipse to the skin region, i.e., extracting 
face as a regular geometric form or ellipse.). 

Regarding Claim 16, Saber et al further disclose the method of extracting the 
specified image subject according to claim 15, wherein said image subject extraction 
processing by said specified image subject extracting algorithm or algorithms is 
performed through dividing it into a plurality of stages (Figures 1-3; Pages 656-657, 
Section 2, Sub-section 2.5, plurality of stages for eye, nose and mouth localization, and 
further plurality of stages based on cost functions Cn^ , Cn^ , Cn^ for eye detection); and 

said image subject extraction processing in a subsequent stage is preferentially 
applied to an extraction area in which said aggregation value in the voting space of 
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said image characteristic quantity exceeds a predetermined value (Figures 1-5; Pages 
656-658, Section 2, Sub-section 2.5, Thresholds ti and t2). 

Regarding Claim 17, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 15, wherein said specified image subject 
extraction processing by said specified image subject extracting algorithm or algorithms 
is performed through dividing it into a plurality of stages (Figures 1-3; Pages 656-657, 
Section 2, Sub-section 2.5, plurality of stages for eye, nose and mouth localization); and 
said image subject extraction processing in a subsequent stage is preferentially applied 
to an extraction area corresponding to said section area for aggregation within a 
preferential frame in the voting space of said image characteristic quantity (Figures 1-5; 
Pages 656-657, Section 2, Sub-section 2.5). 

Regarding Claim 18, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 15, wherein a combination of a plurality of 
image characteristic quantities selected from the group consisting of a position, size, 
direction or orientation of an extraction area and, a posture, density or color tint of an 
image subject is used as the N-dimensional space of said image characteristic quantity 
(Figures 1-5; Pages 656-657, Section 2, Sub-sections 2.1-2.5. Skin/non-skin 
classification and shape classification algorithms implement a plurality of innage 
characteristic quantities consisting of position, size, direction or orientation of an 
extraction area (Face, Eye, Nose and Mouth) and, a posture, density or color tint of an 
image subject (Skin-non-skin color classification in YES color space).). 

Regarding Claim 19, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 15, wherein weighting value lowering 



Application/Control Number: 09/630,31 5 Page 5 

Art Unit: 2623 

processing is applied to a region within a predetermined area on a specific 
cliaracteristic axis with respect to a neighborhood of the region, in which said 
aggregation value became large, in said N-dimensional characteristic stage (Figures 1- 
5; Pages 656-657, Section 2, Sub-sections 2.4-2.5. Principal axes of the elliptical skin 
classified region in N-dimensional eigenspace. Eyes are located on a line which is 
parallel to the minor axis represented by the direction of the eigenvector corresponding 
to the smaller eigenvalues. Finding the minimum of the cost function in the eye region 
on horizontal minor axis is the weighting value lowering processing (Page 657, Formula 
(8)). 

Regarding Claim 20, it is a conventional methodology in statistical decision 
processing to remove a remarkably large size or a remarkably small size from extraction 
data to increase the accuracy and reliability of extracting the subjects for recognition or 
classification purposes (Official Notice). Assigning lower values (sometimes even zero 
value) of weighting factors to the extracted data having large or small size in 
comparison to the mean value of the observed population of the data (outlier data), will 
remove a remarkably large size or a remarkably small size from extraction data. 

With regards to Claim 21 , arguments analogous to those presented for Claim 15 
are applicable to Claim 21 . 

With regards to Claim 22, arguments analogous to those presented for Claim 16 
are applicable to Claim 22. 

With regards to Claim 23, arguments analogous to those presented for Claim 17 
are applicable to Claim 23. 
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With regards to Claim 24, arguments analogous to those presented for Claim 18 
are applicable to Claim 24. 

With regards to Claim 25, arguments analogous to those presented for Claim 19 
are applicable to Claim 25. 

With regards to Claim 26, arguments analogous to those presented for Claim 20 
are applicable to Claim 26. 

Regarding Claim 38, Saber further disclose a method of extracting a specified 
image subject according to Claim 15, wherein said vote comprises an aggregation of 
points indicative of the specified image subject (Section 2.5, centroids of the "holes" 
within the skin segmentation mask). 

With regards to Claim 39, arguments analogous to those presented for Claim 38 
are applicable to Claim 39. 

Regarding Claim 45, Saber et al further disclose a method of extracting a 
specified image subject according to Claim 15, wherein said regular geometric form is 
circular or elliptical (Figures 2 and 3; Page 655, Section 2.). 

With regards to Claim 46, arguments analogous to those presented for Claim 45 
are applicable to Claim 46. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1 , 8, 27, 40, 41 and 43 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Nguyen et al (Segmentation, Grouping and feature Detection for 

Face Analysis; IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4) in view of 

Saber et al (Face Detection and Facial Feature Extraction Using Color, Shape and 

Symmetry-based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 

1015-4651). 

Regarding Claim 1, Nguyen et al disclose a method of extracting a specified 
image subject which successively implements a plurality of specified image subject 
extracting algorithms, comprising the steps of: 

implementing an extracting algorithm of a precedent stage under a 
predetermined extracting condition to obtain an extraction result (Figure 1, connected 
component labeling to create a silhouette and extracting a head image; Page 594, 
Section 2.2); 

changing an extracting condition of a subsequent stage so as to be adapted to 
the thus obtained extraction result (Figure 1 , skin segmentation and face-orientation; 
Page 594, Section 2.2); and 

implementing an extracting algorithm of said subsequent stage under the thus 
changed extracting condition (Figures 3-6, Pages 596-598; first and second columns in 
Figure 3), 

wherein said precedent stage comprises extracting a shape of specified image 
subject (Figure 2, extraction of silhouette and head image). 
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Nguyen et al do not explicitly disclose extracting a shape of regular geometric 
form of the specified image object in precedent stage. 

Saber et al disclose a face detection and facial feature extraction comprising 
extracting a shape of regular geometric form of the specified image object (Page 655, 
Section 2. Skin region has been extracted by fitting an ellipse to the skin region, i.e., 
extracting face as a regular geometric form or ellipse.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Nguyen et al invention according to the teachings of 
Saber et al to extract a shape of regular geometric form of the specified image object in 
precedent stage because the regular geometrical shapes of human faces constitute the 
most accurate features to be utilized as attributes in facial image analysis. 

With regards to Claim 8, arguments analogous to those presented for the 
corresponding limitations of this claim in Claim 1 are applicable to Claim 8. 

With regards to Claim 27, arguments analogous to those presented for the 
corresponding limitation of this claim in Claim 1 are applicable to Claim 27. 

Regarding Claim 40, Nguyen et al further disclose a method of extracting a 
specified image subject according to Claim 1 , wherein said subsequent stage comprises 
detecting a color or hue of the specified image subject (Section 1.3, Color-based 
segmentation, feature detection). 

Regarding Claim 41, Saber et al further disclose a method of extracting a 
specified image subject according to Claim 1 , wherein said regular geometric form is 
circular or elliptical (Figures 2 and 3; Page 655, Section 2.). 
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With regards to Claim 43, arguments analogous to those presented for Claim 41 
are applicable to Claim 43. 

7. . Claims 2-7, 9-14, 32-35, 42 and 44 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nguyen et al (Segmentation, Grouping and feature Detection for 
Face Analysis; IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4) in view of 
Saber eta! (Face Detection and Facial Feature Extraction Using Color, Shape and 
Symmetry-based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 
1 01 5-4651 ) and Ohm! et al (U.S. 5,923,779). 

With regards to Claims 2 and 9, arguments analogous to those presented for 
Claim 1 are applicable to Claim 2 and 9. Saber et al disclose performing a plurality of 
image subject extracting algorithm of the subsequent stage (Sub-section 2.5; Algorithms 
resulted in cost functions indicated by Formulas (5) through (9)). 

Nguyen et al and Saber et al do not explicitly disclose the plurality of image 
subject extracting algorithm of the subsequent stage being performed by parallel 
processing. 

Ohmi et al disclose a face recognition method and corresponding apparatus 
wherein the extracting algorithms for extraction of face parts are perfornned in parallel as 
depicted in Figure 2. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Nguyen et al and Saber et al combination according to 
the teachings of Ohmi et al to perform the plurality of image subject extracting algorithm 
of the subsequent stage by parallel processing because it is an extrennely well known 
procedure routinely implemented in the art to expedite image processing. 
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Regarding Clainn 3, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 2, wherein said respective extraction 
processing conditions are areas to be subjected to extraction processing when 
implementing said plurality of specified image subject extracting algorithms of said 
subsequent stage (Figures 1-5, Face Areas). 

Regarding Claim 4, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 2, wherein said respective extraction 
processing conditions are types of extracting algorithms to be implemented at said 
subsequent stage (Figures 1-3, Pages 656-657, Sub-section 2.4, Shape classification 
algorithms for locating face, nose, mouth, etc.). 

Regarding Claim 5, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 2, wherein said respective extraction 
processing conditions are control parameters inside extracting algorithms to be 
implemented in said subsequent stage (Figures 2 and 3; Pages 656-657, Sub-sections 
2.4 and 2.5, mi, m2, eigenvalues of R, (A,i, A,2), etc.). 

With regards to Claim 6, arguments analogous to those presented for Claim 2 
concerning parallel processing are applicable to Claim 6. 

Saber et al further disclose the method of extracting the specified image subject 
according to Claim 2, wherein said plurality of specified image subject extracting 
algorithms to be implemented in said each stage are of same combination in said 
plurality of stages (Sub-section 2.5. Algorithms resulted in cost functions are indicated 
by Formulas (5) through (10), utilizing the same combination of defining the centroids 
and location of the holes (eyes).). 
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With regards to Claim 7, arguments analogous to those presented for Claim 2 
concerning parallel processing are applicable to Claim 7. 

Saber et a! further disclose the method of extracting the specified image subject 
according to Claim 2, wherein said plurality of specified image subject extracting 
algorithms to be implemented in said each stage are of different combination in said 
plurality of stages (Sub-section 2.5. Algorithms resulted in localization of the nose and 
mouth as depicted in Figure 3.). 

With regards to Claim 9, arguments analogous to those presented for Claim 2 
are applicable to Claim 9. 

With regards to Claim 10, arguments analogous to those presented for Claim 3 
are applicable to Claim 10. 

With regards to Claim 1 1 , arguments analogous to those presented for Claim 4 
are applicable to Claim 1 1 . 

With regards to Claim 12, arguments analogous to those presented for Claim 5 
are applicable to Claim 12. 

With regards to Claim 13, arguments analogous to those presented for Claim 6 
are applicable to Claim 13. 

With regards to Claim 14, arguments analogous to those presented for Claim 7 
are applicable to Claim 14. 

With regards to Claims 32 and 33, arguments analogous to those presented for 
Claims 2 and 9 corresponding to the teachings of Ohmi et al concerning parallel 
processing are applicable to Claims 32 and 33. 
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With regards to Claims 34 and 35, combined teachings of Saber et al and Ohmi 
et al disclose the image subject extraction parallel processing comprises: 

Skin color extraction, face contour extraction, hair-on-head extraction, 
eye/nose/mouth/eyebrow extraction, body extraction (head box in Figure 2 of Ohmi et 
al), and non-background area extraction (Figures 4b and 5b of Saber et al). 

Regarding Claim 42, Saber et al further disclose a method of extracting a 
specified image subject according to Claim 2, wherein said regular geonnetric form is 
circular or elliptical (Figures 2 and 3; Page 655, Section 2.). 

With regards to Claim 44, arguments analogous to those presented for Claim 42 
are applicable to Claim 44. 

8. Claims 28 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nguyen et al (Segmentation, Grouping and feature Detection for Face Analysis; 
IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4) in view of Saber et al 
(Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry- 
based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 1015-4651) 
and Hasegawa et al (Real-time Parallel and Cooperative Recognition of Facial Images 
for an Interactive Visual Human Interface; IEEE Paper ISBN: 1051-4651). 

Regarding Claim 28, Nguyen et al and Saber et al do not explicitly disclose a 
method of extracting a specified image subject according to Claim 1 , wherein said 
predetermined extracting condition comprises electronic flash or backlight information. 

Hasegawa et al disclose a real-time face recognition system comprising 
extraction condition regarding backlight (Page 386, Section 5.1.1, daytime and evening 
lighting conditions). 
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It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Nguyen et al and Saber et al combination according to 
the teachings of Hasegawa et al to consider electronic flash or backlight information as 
one of the feature extraction parameters because it will compensate for environmental 
or ambient brightness changes and will provide more accurate recognition results. 

With regards to Claim 29, arguments analogous to those presented for Claim 28 
are applicable to Claim 29. 

9. Claims 30 and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nguyen et al (Segmentation, Grouping and feature Detection for Face Analysis; 
IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4) in view of Saber et al 
(Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry- 
based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 1015-4651), 
Ohmi et al (U.S. 5,923,779) and Hasegawa et al (Real-time Parallel and Cooperative 
Recognition of Facial Images for an Interactive Visual Human Interface; IEEE Paper 
ISBN: 1051-4651). 

With regards to Claims 30 and 31 , arguments analogous to those presented for 
Claim 28 are applicable to Claims 30 and 31 . 

10. Claims 36 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nguyen et al (Segmentation, Grouping and feature Detection for Face Analysis; 
IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4) further in view of Saber et 
al (Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry- 
based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 1015-4651). 
Ohmi et al (U.S. 5,923,779) and Wurtz (Object Reconstruction Robust Under 
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Transitions, Deformations, and Changes in Background", IEEE Transactions on Pattern 
Analysis and Machine Intelligence, ISBN: 0162-8828). 

Nguyen et al, Saber et a! and Ohmi et al do not explicitly disclose the method of 
extracting a specified image subject wherein the extraction algorithms comprise 
different degrees of resolution. 

Wurtz discloses a face recognition comprising extraction algorithms at different 
degrees of resolution (Sections 2.1 and 2.2). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Saber et al and Ohmi et al combination according to the 
teachings of Wurtz to utilize extraction algorithms at different degrees of resolution 
because it will discard low level features due to a variation in background (Wurtz; 
Abstract and Section 2.4). 

Contact Information 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mehrdad Dastouri whose telephone number is (703) 
305-2438. The examiner can normally be reached on Monday to Friday from 8:00 a.m. 
to 4:30 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
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Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-ciirect.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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